Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more strict verifications when user provides a platform #336

Merged
merged 2 commits into from
Jan 8, 2025

Conversation

wagoodman
Copy link
Contributor

@wagoodman wagoodman commented Jan 4, 2025

Today we allow for users to specify a platform for images being resolved. The expectation is that if the fetched/resolved image does not match the requested platform then we should error out and not continue. Bugs were found in both how the Docker daemon provider (thus Podman as well since this is shared code) as well as the OCI registry provider. Though the bugs were slightly different, at a high level they were the same: a user could provide a platform to fetch and the provider in some cases would ignore this platform and fetch the image for a different platform.

For the Docker daemon provider when monitoring the pull event status JSONL stream the Error field was not being considered, which is where this kind of error is raised up.

For the OCI registry provider, passing the remote.WithPlatform option only sets the platform field on the v1.Descriptor object, but does not have any effect on what is fetched in the case where there is no manifest list or index (there is a single architecture in the registry). In the case of manifest list or index the correct platform was being resolved. The change here was to explicitly pull the container config and validate against the os and architecture fields (which are required via the OCI spec).

An additional step within the OCI registry provider is checking if the manifest is a list/index or a single manifest. If the user does NOT give a platform then the current default behavior for this provider is to set one based off of linux/<GOARCH>. The intended behavior is to honor what the single-architecture manifest instead of overriding with a default value. This PR enforces this intended behavior by clearing the platform options after fetching the manifest and checking the MediaType for single vs multi arch support.

The last change is to allow for providers to express errors that indicate that the image was fully resolved by the provider but there was a transient or fundamental error. This new error is intended to signal to the caller that no further providers in a set of providers should be attempted, and to raise up this single error. For instance, if we tried 7 providers with 7 "not found" errors and we reach the Docker provider and it raises up a platform mismatch error, we don't want to attempt the Registry provider since we expect a similar outcome. This fosters a fail-fast approach for cases we know the user should be paying attention to (and not get lost in a pile of errors from multiple providers). Instead a new specific error type has been added to allow the caller to more clearly reason about platform mismatch errors.

@wagoodman wagoodman added the bug Something isn't working label Jan 4, 2025
@wagoodman wagoodman requested a review from a team January 4, 2025 03:22
@wagoodman wagoodman self-assigned this Jan 4, 2025
Copy link

github-actions bot commented Jan 4, 2025

Benchmark Test Results

Benchmark results from the latest changes vs base branch
make .tool/task
make[1]: Entering directory '/home/runner/work/stereoscope/stereoscope'
make[1]: Leaving directory '/home/runner/work/stereoscope/stereoscope'
.tool/task show-benchstat
?   	github.com/anchore/stereoscope	[no test files]
?   	github.com/anchore/stereoscope/examples	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal	0.004s
?   	github.com/anchore/stereoscope/internal/bus	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal/containerd	0.007s
PASS
ok  	github.com/anchore/stereoscope/internal/docker	0.005s
?   	github.com/anchore/stereoscope/internal/log	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal/podman	0.005s
?   	github.com/anchore/stereoscope/pkg/event	[no test files]
?   	github.com/anchore/stereoscope/pkg/event/parsers	[no test files]
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkTarIndex-4   	   33673	     35788 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33488	     35713 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33549	     35490 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   31338	     35672 ns/op	    5699 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33390	     35559 ns/op	    5699 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33636	     35740 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33691	     35674 ns/op	    5700 B/op	      93 allocs/op
PASS
ok  	github.com/anchore/stereoscope/pkg/file	10.873s
PASS
ok  	github.com/anchore/stereoscope/pkg/filetree	0.005s
?   	github.com/anchore/stereoscope/pkg/filetree/filenode	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/image	0.005s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/containerd	0.008s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/docker	0.005s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/oci	0.005s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/oci/credhelpers	0.005s
?   	github.com/anchore/stereoscope/pkg/image/podman	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/image/sif	0.004s
?   	github.com/anchore/stereoscope/pkg/imagetest	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/tree	0.003s
PASS
ok  	github.com/anchore/stereoscope/pkg/tree/node	0.003s
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/test/integration
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkSimpleImage_GetImage/docker-archive-4 	     982	   1221953 ns/op	  280776 B/op	    2495 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	     972	   1228493 ns/op	  280561 B/op	    2494 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	     979	   1249531 ns/op	  280360 B/op	    2494 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	     814	   1311221 ns/op	  280231 B/op	    2494 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	     970	   1261907 ns/op	  280051 B/op	    2493 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	     982	   1210668 ns/op	  280036 B/op	    2493 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	     973	   1300201 ns/op	  279852 B/op	    2493 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      63	  18314412 ns/op	  402977 B/op	    2698 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      61	  18700131 ns/op	  401395 B/op	    2696 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      58	  18652750 ns/op	  402437 B/op	    2698 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      61	  19003733 ns/op	  401997 B/op	    2697 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      61	  19123214 ns/op	  402238 B/op	    2697 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      57	  18789789 ns/op	  402654 B/op	    2697 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      62	  18765848 ns/op	  401225 B/op	    2697 allocs/op
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 345B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load build context
#3 transferring context: 209B done
#3 DONE 0.0s

#4 [1/3] ADD file-1.txt /somefile-1.txt
#4 CACHED

#5 [2/3] ADD file-2.txt /somefile-2.txt
#5 CACHED

#6 [3/3] ADD target /
#6 CACHED

#7 exporting to image
#7 exporting layers done
#7 writing image sha256:5a8f3fe3fce58de04d3e09f11ab85bb9213d816c319ec8031f13af86e5265856 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:latest done
#7 DONE 0.0s
ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
--- FAIL: BenchmarkSimpleImage_GetImage
    image_fixtures.go:193: using existing image tar: 'test-fixtures/cache/stereoscope-fixture-image-simple-04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7.tar' (size: 22528, modified: 2025-01-08 16:38:20.071767799 +0000 UTC, mode: -rw-r--r--)
    image_fixtures.go:241: Build docker image: name="stereoscope-fixture-image-simple" tag="04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7"
    image_fixtures.go:291: saveImage running: docker image save stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7
    image_fixtures.go:286: 
        	Error Trace:	/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:286
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:162
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:152
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:33
        	            				/home/runner/work/stereoscope/stereoscope/test/integration/fixture_image_simple_test.go:163
        	Error:      	Received unexpected error:
        	            	exit status 1
        	Test:       	BenchmarkSimpleImage_GetImage
        	Messages:   	could not import docker image to containerd (shell out)
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53944	     22222 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53812	     22270 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53716	     22211 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53365	     22244 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53806	     22294 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53604	     22262 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53694	     22243 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53504	     22212 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53766	     22227 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53995	     22287 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53570	     22182 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53443	     22424 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53551	     22285 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53508	     22350 ns/op	    2712 B/op	      21 allocs/op
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 345B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load build context
#3 transferring context: 209B done
#3 DONE 0.0s

#4 [1/3] ADD file-1.txt /somefile-1.txt
#4 CACHED

#5 [2/3] ADD file-2.txt /somefile-2.txt
#5 CACHED

#6 [3/3] ADD target /
#6 CACHED

#7 exporting to image
#7 exporting layers done
#7 writing image sha256:5a8f3fe3fce58de04d3e09f11ab85bb9213d816c319ec8031f13af86e5265856 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:latest done
#7 DONE 0.0s
ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
--- FAIL: BenchmarkSimpleImage_FetchSquashedContents
    image_fixtures.go:193: using existing image tar: 'test-fixtures/cache/stereoscope-fixture-image-simple-04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7.tar' (size: 22528, modified: 2025-01-08 16:38:20.071767799 +0000 UTC, mode: -rw-r--r--)
    image_fixtures.go:241: Build docker image: name="stereoscope-fixture-image-simple" tag="04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7"
    image_fixtures.go:291: saveImage running: docker image save stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7
    image_fixtures.go:286: 
        	Error Trace:	/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:286
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:162
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:152
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:33
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:64
        	            				/home/runner/work/stereoscope/stereoscope/test/integration/fixture_image_simple_test.go:189
        	Error:      	Received unexpected error:
        	            	exit status 1
        	Test:       	BenchmarkSimpleImage_FetchSquashedContents
        	Messages:   	could not import docker image to containerd (shell out)
FAIL
exit status 1
FAIL	github.com/anchore/stereoscope/test/integration	38.869s
?   	github.com/anchore/stereoscope/test/integration/test-fixtures/registry	[no test files]
FAIL
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
ctr: 
           │ .tmp/benchmark-c871710.txt │
           │           sec/op           │
TarIndex-4                  35.67µ ± 1%

           │ .tmp/benchmark-c871710.txt │
           │            B/op            │
TarIndex-4                 5.566Ki ± 0%

           │ .tmp/benchmark-c871710.txt │
           │         allocs/op          │
TarIndex-4                   93.00 ± 0%

pkg: github.com/anchore/stereoscope/test/integration
                                      │ .tmp/benchmark-c871710.txt │
                                      │           sec/op           │
SimpleImage_GetImage/docker-archive-4                  1.250m ± 5%
SimpleImage_GetImage/podman-4                          18.77m ± 2%
geomean                                                4.842m

                                      │ .tmp/benchmark-c871710.txt │
                                      │            B/op            │
SimpleImage_GetImage/docker-archive-4                 273.7Ki ± 0%
SimpleImage_GetImage/podman-4                         392.8Ki ± 0%
geomean                                               327.9Ki

                                      │ .tmp/benchmark-c871710.txt │
                                      │         allocs/op          │
SimpleImage_GetImage/docker-archive-4                  2.494k ± 0%
SimpleImage_GetImage/podman-4                          2.697k ± 0%
geomean                                                2.594k

ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
                                                   │ .tmp/benchmark-c871710.txt │
                                                   │           sec/op           │
SimpleImage_FetchSquashedContents/docker-archive-4                  22.24µ ± 0%
SimpleImage_FetchSquashedContents/podman-4                          22.29µ ± 1%
geomean                                                             22.26µ

                                                   │ .tmp/benchmark-c871710.txt │
                                                   │            B/op            │
SimpleImage_FetchSquashedContents/docker-archive-4                 2.648Ki ± 0%
SimpleImage_FetchSquashedContents/podman-4                         2.648Ki ± 0%
geomean                                                            2.648Ki

                                                   │ .tmp/benchmark-c871710.txt │
                                                   │         allocs/op          │
SimpleImage_FetchSquashedContents/docker-archive-4                   21.00 ± 0%
SimpleImage_FetchSquashedContents/podman-4                           21.00 ± 0%
geomean                                                              21.00
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
ctr: 
           │ .tmp/benchmark-c871710.txt │
           │           sec/op           │
TarIndex-4                  35.67µ ± 1%

           │ .tmp/benchmark-c871710.txt │
           │            B/op            │
TarIndex-4                 5.566Ki ± 0%

           │ .tmp/benchmark-c871710.txt │
           │         allocs/op          │
TarIndex-4                   93.00 ± 0%

pkg: github.com/anchore/stereoscope/test/integration
                                      │ .tmp/benchmark-c871710.txt │
                                      │           sec/op           │
SimpleImage_GetImage/docker-archive-4                  1.250m ± 5%
SimpleImage_GetImage/podman-4                          18.77m ± 2%
geomean                                                4.842m

                                      │ .tmp/benchmark-c871710.txt │
                                      │            B/op            │
SimpleImage_GetImage/docker-archive-4                 273.7Ki ± 0%
SimpleImage_GetImage/podman-4                         392.8Ki ± 0%
geomean                                               327.9Ki

                                      │ .tmp/benchmark-c871710.txt │
                                      │         allocs/op          │
SimpleImage_GetImage/docker-archive-4                  2.494k ± 0%
SimpleImage_GetImage/podman-4                          2.697k ± 0%
geomean                                                2.594k

ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
                                                   │ .tmp/benchmark-c871710.txt │
                                                   │           sec/op           │
SimpleImage_FetchSquashedContents/docker-archive-4                  22.24µ ± 0%
SimpleImage_FetchSquashedContents/podman-4                          22.29µ ± 1%
geomean                                                             22.26µ

                                                   │ .tmp/benchmark-c871710.txt │
                                                   │            B/op            │
SimpleImage_FetchSquashedContents/docker-archive-4                 2.648Ki ± 0%
SimpleImage_FetchSquashedContents/podman-4                         2.648Ki ± 0%
geomean                                                            2.648Ki

                                                   │ .tmp/benchmark-c871710.txt │
                                                   │         allocs/op          │
SimpleImage_FetchSquashedContents/docker-archive-4                   21.00 ± 0%
SimpleImage_FetchSquashedContents/podman-4                           21.00 ± 0%
geomean                                                              21.00

@wagoodman wagoodman force-pushed the fix-platform-options branch from a7b731e to 9c7e279 Compare January 4, 2025 04:40
pkg/image/provider.go Outdated Show resolved Hide resolved
@wagoodman wagoodman enabled auto-merge (squash) January 8, 2025 16:38
@wagoodman wagoodman merged commit e394cf5 into main Jan 8, 2025
7 checks passed
@wagoodman wagoodman deleted the fix-platform-options branch January 8, 2025 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants